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r\] • We consider the problem of computing the rank of an tti x n matrix A over a field. We present a 

randomized algorithm to find a set of r = rank(j4) linearly independent columns in 0(|^| + r") field 
operations, where |^| denotes the number of nonzero entries in A and uj < 2.38 is the matrix multiplication 
exponent. Previously the best known algorithm to find a set of r linearly independent columns is by 

I I ' Gaussian elimination, with running time 0{mnr"~^). Our algorithm is faster when r < max{7n, n}, 

jyj , for instance when the matrix is rectangular. We also consider the problem of computing the rank of a 

^ ' matrix dynamically, supporting the operations of rank one updates and additions and deletions of rows 

and columns. We present an algorithm that updates the rank in 0{mn) field operations. We show that 
these algorithms can be used to obtain faster algorithms for various problems in numerical linear algebra, 
combinatorial optimization and dynamic data structure. 
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t^ ■ 1 Introduction 

^-». ' Given an m x n matrix A over a field F, the rank of A, denoted by rank(A), is the maximuni number of 

fvq . linearly independent columns of A. We consider the problem of computing rank(A) and finding a set of 

rank(A) linearly independent columns efficiently. It is a basic computational problem in numerical linear 
algebra that is used as a subroutine for other problems [IHl HO] • It also has a number of applications in graph 
algorithms and combinatorial optimization: Some of the fastest algorithms for graph matching J331I22J . graph 
connectivity [TOl [35j [12] , matroid optimization problems [22l [11] are based on fast algorithms for computing 
matrix rank and finding linearly independent columns. 

The traditional approach to compute rank(A) is by Gaussian elimination. For an m x n matrix with m < n, 
it is known that this approach can be implemented in 0{nm'^^^) field operations J6l[27], where ui < 2.38 is 
the matrix multiplication exponent |14j . More generally, given an m x n matrix and a parameter k < m < n, 
one can compute min{rank(A), k} in 0{n'mk"~^) field operations |39j . The Gaussian elimination approach 
has the advantage that it can also find a set of min{rank(A) , k} linearly independent columns in the same 
time. These algorithms are deterministic. 

There are also randomized algorithms to compute the value of rank(j4) more efficiently. We know of three 
approaches. 

1. The first approach is to do an efficient preconditioning [29l[9]. Let B = T1AT2 where Ti and T2 are 
Toeplitz matrices with entries chosen uniformly and randomly from a large enough subset of the field. 
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Then B can be computed in 0{mn) time because of the structure of Ti and T2. Let r — rank(A). It 
is proven that [29] the leading r x r minor of B is of full rank with high probability. Thus rank(^) 
can be computed in 0{mn + r") field operations. There is another efficient prcconditioncr based on 
butterfly network [p] with similar property and running time. This approach works for any field. 

2. There is a black-box approach that computes rank(yl) in 0{m ■ \A\) field operations [43l[20l[37] where 
1^1 is the number of nonzero entries of A. The method is based on computing the minimal polynomial 
of A for Krylov subspaces. It does not require to store A explicitly, as long as there is an oracle to 
compute Ab for any vector b. This approach is fast when the matrix is sparse, and it works for any 
field. 

3. Another approach is based on random projection for matrices over real numbers. Given anmxn matrix 
A over M, one can reduce A into an m x (to log m) matrix A' so that rank(A) = rank[A') with high 
probability [36] by the Johnson-Lindenstrauss lemma. The matrix A' can be computed efficiently using 
fast Johnson-Lindenstrauss transform [5| ISl, and this implies an 0{nm + m^) randomized algorithm 
to compute rank(yl). This approach is only known to work for matrices over real numbers. 

We remark that only the Gaussian elimination approach can also find a set of rank(A) linearly independent 
columns, while other approaches can only compute the value of rank(j4). 

1.1 Main Results 

We present faster randomized algorithms to compute matrix rank and show their applications. In this section 
we use the O notation to hide (small) polylog factors in the time bounds. We will state the precise time 
bounds in the technical sections. We assume that there is at least one nonzero entry in each row and each 
column, and thus \A\ > max{7Ti, n}. 

Theorem 1.1. Given an m x n matrix A over a field F and a parameter k where k < m < n, there 
is a randomized algorithm to compute min{rank(yl) , fc} in 0{\A\ -I- min{fc", fc|yl|}) field operations where 
\A\ denotes the number of nonzeros in A. Furthermore, there is a randomized algorithm to find a set of 
min{rank(A), k} linearly independent columns in 0{\A\ + k'^) field operations. 

For computing min{rank(A), fc}, previous algorithms require 0{mn + k'^) field operations, while we replace 
the mn term by \A\ and remove the (small) polylog factor. More importantly, we can also find a set of 
min{rank(A), fc} linearly independent columns in about the same time, which is considerably faster than 
the 0{mnk'^~^) algorithm by Gaussian elimination when k is small. For instances, we can find a set of 
k ~ n^l^ w n'^''^ linearly independent columns in 0(|yl|) field operations, and a set of fc = 72i/("-i) ^ 
j^o.72 lij^gaj-jy independent columns in 0{mn) field operations, while previously it was possible only for 
k — 0(polylog(n)). The algorithm for finding linearly independent columns is needed in all applications of 
Theorem ILU that we will describe in the next subsection. 



We also present a dynamic algorithm to efficiently update the matrix rank. 

Theorem 1.2. Given anmxn matrix A over a field F , there is a randomized algorithm to compute rank(A) 
dynamically in 0(rnn) field operations in the worst case, supporting the operations of rank one updates and 
adding and deleting rows and columns. 

Previously there is a dynamic algorithm to update the matrix rank in 0{n^) field operations for an n x n 
square matrix, supporting the operation of rank one updates \W[ I35j . There are also subquadratic dynamic 
algorithms to update the matrix rank when few entries are changed [16[ I35j . Our algorithm supports the 
new operations of adding and deleting rows and columns. These new operations will be useful in computing 
graph connectivities dynamically (see Theorem ll.5|) . 
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Table 1.1: Time complexity of algorithms for some problems in combinatorial optimization 

1.2 Applications 

The matrix rank algorithms can be readily applied to various problems in numerical linear algebra, com- 
binatorial optimization, and dynamic data structure. First we show that the algorithms can be applied to 
computing a rank-one decomposition, finding a basis of the null space, and performing matrix multiplication 
for a low rank matrix. 

Theorem 1.3. Let A be an m x n matrix over a field F. Let r = rank(y4). Let m' = min{r7z,Ti}. Let 
uj{a, b, c) be the exponent for multiplying an n° x n'^ matrix with an n^ x n"^ matrix. 

1. There is a randomized algorithm to compute an m x r matrix X and an r x n matrix Y such that 
A^ XY md{\A\ +r'^(ia,iog,,m')) ^ d{\A\ +m'r'^-^) steps. 

2. There is a randomized algorithm to find a basis of the null space of A in 0{\A\ + r'^(i'i'i°gj-")) = 
d{\A\ +nr'^^^) steps. 

3. Let A and B be nxn matrices. There is a randomized algorithm to compute AB in 0{n'^^^°^^'^'^'^') = 
d{n^r'^-'^) steps. 

The success probability for all three tasks is at least 1 — 0{\og{nm)/\A\^'^). 

Previously the best known algorithms require Q{mnr'^^^) for the first two tasks, and Q{n'^r'^^'^) for the 
third task. Our algorithms are faster than the existing algorithms, especially when r is small. For rank-one 
decomposition, the algorithm requires only 0{mn) field operations when r < (max{7Ti,ri})°'^^. For finding 
null space, the algorithm requires only 0{mn) field operations when r < to*^'^^. For matrix multiplication, 
the algorithm requires only 0{n^'^'^) field operations when r < n^-^^, since w(0.29, 1,1) < 2 + e for any 
e > [13l [26] . The statement about matrix multiplication essentially says that the problem of multiplying 
two nxn matrices while one matrix is of rank r can be reduced to the problem of multiplying an r x n 
matrix and an n x rt matrix. 

In combinatorial optimization, there are algebraic formulations of the problems that relate the optimal value 
to the rank of an associated matrix. Using this connection, we can apply the algorithm in Theorem 11.11 
to obtain fast algorithms for graph matching and matroid optimization problems. See Section |4] for the 
definitions of these problems. 

Theorem 1.4. Let opt be the optimal value of an optimization problem. 

1. Given an undirected graph G = {V,E), there is a randomized algorithm to find a matching of size 
minjopt, fc} in d{\E\ + fc") time. 

2. Given a linear matroid intersection problem or a linear matroid parity problem with an r x 2n matrix 
A, there is a randomized algorithm to find a solution of size minjopt, k} in 0(|j4| -|- nk^^^) time. 

3. Given a linear matroid union problem with an r xn matrix \A\, there is a randomized algorithm to find 
minjopt, fc} disjoint bases in 0{k\A\ +rmTi{k"'^^b'^ ,k^b^}) time, where b denotes the size of a basis. 



Table 11.11 lists the time complexity of the best known combinatorial algorithms and algebraic algorithms for 
these problems. Notice that previous algebraic algorithms have the same time complexity even when the 
optimal value is small. On the other hand, combinatorial algorithms for these problems are based on finding 
augmenting structures iteratively, and thus the number of iterations and the overall complexity depend on 
the optimal value. While the previous algebraic algorithms are faster than combinatorial algorithms only 
when the optimal value is large, the results in Theorem 11.41 show that the algebraic approach can be faster 
for any optimal value. For the matroid optimization problems, the algorithms in Theorem [L4] are faster than 
previous algorithms in any setting. The result in the graph matching problem can be applied to the subset 
matching problem 4, and the lopsided bipartite matching problem |8j . See Section 2] for more discussions 
on previous work for these problems. 



The dynamic matrix rank algorithm in Theorem ll.2l can be applied to obtain a dynamic algorithm to compute 
edge connectivities in a directed graph. 

Theorem 1.5. Given an uncapacitated directed graph G = {V,E), there is a randomized algorithm to com- 
pute all-pairs edge-connectivities dynamically in 0{\E\'^) time and 0{\E\'^) space, supporting the operations 
of adding and deleting 



In undirected graphs, there are polylogarithmic time dynamic algorithms for computing fc-edge-connectivity 
for k < 2 [24], and a 0(|y||i?|) time algorithm to compute all pairs edge connectivities [5]. The corresponding 
problems are more difficult for directed graphs. There is a subquadratic dynamic algorithm for computing 1- 
edge-connectivity in directed graphs [34] . For all pairs edge connectivities in directed graphs, we do not know 
of any dynamic algorithm that is faster than the straightforward dynamic algorithm that uses 0(|yp|i?|) 
time and 0(|yp|-E|) space, by storing the flow paths for each pair and running an augmentation step for each 
pair after each edge update. For graphs with 0(|\^|) edges (e.g. planar graphs), the amortized complexity 
of our algorithm to update the edge connectivity for one pair is 0(1) field operations. 

1.3 Methods 

Similar to the preconditioning approach, our approach is to compress the matrix A into a 0{k) x 0{k) matrix 
B, while keeping the property that min{rank(i?), k} = min{rank(j4), k} with high probability. To illustrate 
the ideas, we consider the special case of computing the rank of a rectangular mx n matrix A where m < n, 
and the goal is to compress the matrix into an to x 0[m) matrix with rank(_B) — rank(j4). We present two 
efficient methods to do the compression, assuming the field size is sufficiently large for now. The first method 
is inspired by the random linear coding algorithm [23j in network coding J^ and its efficient implementation 
using superconcentrators [12j . Suppose we write each column of i? as a random linear combination of all 
the columns of A. Then it can be shown that rank(i3) — rank(v4) with high probability by the Schwartz- 
Zippel lemma, but the direct implementation of this method requires a fast rectangular matrix multiplication 
algorithm. To do the compression efficiently, we use a construction similar to that of magical graphs [25| in 
the construction of superconcentrators. We prove that if each column of i? is a random linear combination of 
0{n/m) random columns of A, it still holds with high probability that rank(i?) = rank(A). In addition, this 
property still holds when each column of A is involved in only 0(1) linear combinations, and so the sparsity 
of the matrix can be preserved, i.e. \B\ = 0{\A\). Hence B can be constructed in 0(|yl|) field operations, 
and rank(i3) can be computed in 0{m'^) field operations, and thus rank(A) can be computed in 0(|A| +7n") 
field operations. Based on a bounded degree condition of the magical graphs, the above procedure can be 
applied iteratively to reduce the number of columns of A progressively, so that a set of rank(A) linearly 
independent columns in A can be found in 0(|A| + to") field operations. 

Another method to compute B is to multiply A with an n x ?7i random Vandermonde matrix V with only 
one variable. We show that rank(_B) = rank(^) with high probability, by using the Cauchy-Binet formula 



and a base exchange argument. The m x m matrix B = AV can be computed in 0{mn) field operations 
using fast Fourier transform. This provides an alternative way to compute rank(j4) in 0{mn + m") field 
operations, although it is slower than the above method. The advantage of this method is that it allows us 
to update the matrix rank efficiently when we add and delete rows and columns of A, because of the special 
structures of the Vandermonde matrices. For instance, when the m x n matrix A is changed from to < n to 
m > n, we can change the representation from B = AV to B' = V A by doing an inverse Fourier transform. 
This allows us to update rank(yl) in 0{mn) field operations in the worst case. 

2 Fast Matrix Rank Algorithms 

In this section we will prove Theorem ll.il Let Ahe ai\ mx n matrix over a field F . We will assume that A 
is given by a list of the value and the position of its non-zero entries, and each row and column of A contains 
at least one non-zero entry, so |^| > max(ri,m). We will also assume that |F| = rj(n^) by the following 
lemma using standard techniques. 

Lemma 2.1. Let A be anmx n matrix over a field F with p'^ elements. We can construct a finite field F' with 
p'^^ = J7(n*) elements and an injective mapping f : F —>■ F' so that the image of F is a subfield of F' . Then 
the mx n matrix A' where a'^ = fi'^ij) satisfies the property that rank(j4') = rank(j4). This preprocessing 
step can be done in 0{\A\) field operations. Each field operation in F' can be done in 0(log |_F| +\ogn) steps. 

Proof. All the statements in this proof refer to the statements in book 20 . Let q = p'^. By Theorem 14.42 
in [20] , we can construct a monic irreducible polynomial h with degree k in expected 0{k^ log^ k log log fc(log k+ 
logg)) field operations in Fq. Note that the collection of polynomials with coefficients in Fq and degree less 
than k, with multiplications and division under modulo h, is a field with size q'' . So we can use an ordered 
fc-tuple (co,ci, . . . , Cfc_i) with Ci € Fq to represent an element J2i=o ^i^^ i^ ^q''- The injective mapping / 
in the statement is just the identity mapping in this construction, i.e. /(c) = (c, 0,0, . . . , 0). The overall 
preprocessing time is 0(|A| + fc^ log^ fcloglogfc(logfc + log 9)) = 0(|^|) field operations in Fqh. It follows 
directly that rank(^') ~ rank(j4). 

Additions and subtractions are done coordinate-wise, and thus requires 0{k) field operations. For two poly- 
nomials gi and 52 with coefficients in Fq and degree less than fc, gi x g2 can be computed in 0{k log k log log k) 
field operations in Fq, by Theorem 8.22 and Exercise 8.30 in [3D]. So gi x 52 mod h can be computed in 
0(fc log fcloglogfc) field operations in Fq by Theorem 9.6 in [201. Division a/b is done by multiplying the in- 
verse a X b~^. The inverse b~^ can be computed by the extended euclidean algorithm, in 0(fclog fcloglogfc) 
field operations in Fq by Theorem 11.7 in [30]. Since field operations in Fq can be computed in O(logq) 
steps, the operations in Fqk in our representation can be done in 0(logq'^), where O hides some polylog 
factors of log q^ . D 

Suppose a parameter k is given and the task is to compute min{rank(A), k}. Our approach is to compress the 
matrix into a 0{k) x 0{k) matrix whose rank is at least min{rank(A), k} with high probability. Our method 
is inspired by the random linear coding algorithm |23i 112] in network coding [1] . We can construct an m x k 
matrix B where each column of i? is a random linear combination of the columns of A, i.e. Bi — X^^i ^j,i-^j 
where Ai and Bi denote the i-th column of A and B respectively and Cj^i is uniformly independent random 
element in i^. In other words, B = AC where C is an n x fc matrix where each entry is a random element 
in F. It can be shown that rank(i?) = rank(A) with high probability using the Schwartz-Zippel lemma 
(see Lemma l2.3p . but it requires a fast rectangular matrix multiplication algorithm :26j to compute B. We 
observe that this way of constructing B is the same as doing the random linear coding algorithm in a single 
vertex with n incoming edges and k outgoing edges. And so the ideas of using a superconcentrator to do 
the random linear coding efficiently [12] can be applied to construct an to x fc matrix B' in 0{mn) field 



operations, while rank(B') = rank(^) with high probabihty (see Appendix 1X1 for details). This implies 
that min{rank(A), k} can be computed in 0{mn + fc") field operations with high probability, improving the 
existing algorithms by removing the polylog factor. There are, however, two disadvantages of this method. 
One is that the compression algorithm requires Q{mn) field operations even when A is a sparse matrix. 
Another is that we do not know how to find a set of min{rank(A), k} linearly independent columns of A 
using this method. 

To improve the compression algorithm, we use sparse random bipartite graphs similar to that of magical 
graphs [5S] in the construction of superconcentrators. The idea is to choose a sparse matrix C so that 
B — AC can be computed efficiently while min{rank(i?), k} — min{rank(^), k} with high probability, but it 
is easier to explain the compression algorithm using graph theoretical concepts. Our construction requires a 
probability distribution of bipartite graphs with the following properties. 

Definition 2.2 (Magical Graph). A random bipartite graph G — {X,Y;E) is {k,e) -magical if for every 
subset S '^ X with \S\ = k, the probability that there is a matching in G in which every vertex in S is 
matched is at least 1 — e. 

Notice that this definition only requires a particular subset S of size k can be matched to the other side with 
high probability, while the definition in 25^ requires that all subsets up to certain size can be matched to 
the other side. In the following we first see how to use a magical graph to do compression (Figure [2. ip . and 
then we will see how to generate a magical graph with good parameters efficiently. 



Input: (1) An m x n matrix A over a field F. 

(2) A (fc, e)-magical graph G = {X, Y; E) with \X\ = n and |y| = /. 
Output: An m x I matrix B over a field F with min{rank(i3), fc} — min{rank(A), fc}. 

Algorithm: Let X — {xi, . . . , x„} and Y — {j/i, . . . , y/}. Each column of A corresponds to a vertex in X 
and each column of B corresponds to a vertex in Y . Let Aj be the j-th column of A for 1 < j < n and Hi 
be the i-th column of B for 1 < i < I. Construct B by writing Bi as a random linear combination of those 
columns of ^ whose corresponding vertices have an edge to yi. More precisely, we write Bi = X]e=a: viSS c-eAj 
for 1 < i < ^ where Cg is an independent uniformly random element in F for each edge e € G. 



Figure 2.1: Compression Algorithm by Magical Graphs. 

Lemma 2.3. The probability that the algorithm in Fiaure lKT] returns a matrix B such that niin{rank(i3), k} = 
niin{rank(y4) , k} is at least 1 — e — k/\F\. 

Proof. Clearly rank(i3) < rank(A) since the column space of i? is a subspace of the column space of A. 
So niin{rank(_B), fc} < nfin{rank(yl), fc}, and it remains to show that rank(i?) > min{rank(A), fc} with high 
probability. 

Let k' — min{rank(y4), k}. Let S" be a set of linearly independent columns of A with \S\ — k' , and let Ajj^s 
he a k' X k' submatrix of A with rank(j4j/ 5) = k' . Since G is a magical graph, the probability that there is 
a matching M in which every vertex in S is matched is at least 1 — e. Suppose such a matching M exists 
and let T be the neighbors of 5' in M with \T\ = \S\ = k' . If we view each Cg as a variable, then det{Bu,T) 
is a multivariate polynomial with total degree k' . By setting Cg = 1 for each e € M and Ce — for each 
e e _E — M, we get that Bu,t = Au^s and thus det (i?[/,T) is a nonzero multivariate polynomial since Ajj^s 
is of full rank. By the Schwartz-Zippel lemma, if we substitute each variable Cg by a random element in a 
field F, then the probability that det{Bu,T) = is at most k' /\F\ < k/\F\. So, if G has a matching that 
matches every vertex in S, then rank(i3) > ra.nk{BijT) = k' with probability at least 1 — k/\F\. Therefore 
the algorithm succeeds with probability at least 1 — e — k/\F\. D 



We show that a magical graph with good parameters can be generated efficiently. 

Lemma 2.4. There is a randomized 0{\X\) time algorithm to construct a (k,0{l/k)) -magical graph G = 
[X^Y'.E) for any values of \X\ > \Y\ > ck where c > 11, with the additional properties that each vertex of 
X is of degree 2 and each vertex of Y is of degree at most 2\\X\/\YW . 

We note that the magical graphs in |25j cannot be used directly because of the following reasons: (1) the 
failure probability in 25 is a constant while we need a much smaller failure probability in order to find a 
set of linearly independent columns, (2) we need the additional property that the graph is almost regular 
to find a set of linearly independent columns. Also the proof is somewhat different and the degree of the 
vertices in X is smaller. 

Proof. The generation algorithm is simple. We assume that \X\ is a multiple of \Y\\ otherwise we construct 
a slightly larger graph and delete the extra vertices. We first construct a 2-regular graph G" with \X\ vertices 
on both sides, by taking the union of two random perfect matchings independently from \X\ vertices to \X\ 
vertices. Then we divide the \X\ vertices on one side into \Y\ groups where each group has |X|/|y| vertices. 
We obtain G by merging each group into a single vertex, and so each vertex in Y is of degree 2|X|/|F|. 

For any S" C X with \S\ — fc, we analyze the probability that there is a matching in G in which every vertex 
in S is matched. By Hall's theorem, we need to show that for any S' C S, the neighbor set of S' in G is of 
size at least \S'\. To analyze the probability that the neighbor set of S' is at least \S'\ for a fixed S' C 5, we 
consider the equivalent random process where the 2|S"| edges incident on S" are added one by one. Consider 
the i-th edge added. We say that it is a bad edge if the other endpoint falls in the same group with some 
previously added edges. If the neighbor set size of S' is less than |S"|, then there must be at least |S"| + 1 bad 
edges out of the 2|S"| edges, and the probabihty that an edge is bad is less than |S"|/|y|. So the probabihty 
that the neighbor set size of S" is less than |5'| is less than 

S'\ + l) \\Y\) 

by a union bound on the possible |5'| + 1 bad edges. Summing over the choices of the size of S' and the 
choices of S' with that size, we have that the probability that there is a subset S' C S with less than |5'| 
neighbors is at most 

k 



< 



^oUJ Ck 



= 0(l/fc), 

using \Y\ > ck and the identity X]^o ''^ ' ^ ~ ^/(^ ~ '')^ ^'^^ r < 1, and setting r = 4e/c as c > 11 > 4e 
by our assumption. Therefore, by Hall's theorem, the probability that there is a matching in which every 
vertex in S is matched is at least 1 — 0(l/fc). D 



We can combine Lemma 12.41 and Lemma 12.31 to obtain an efficient compression algorithm. 

Theorem 2.5. Suppose an m x n matrix A over a field F is given. Given k, there is an algorithm that 
constructs an m x 0{k) matrix B over F with the following properties. 

1. min{rank(A), fc} = min{rank(i?), fc} with probability at least 1 — 0{l/k) — 0{k/\F\). 



2. \B\ = 0(|A|) and B can he constructed in 0{\A\) field operations. 

Proof. We can assume n > life; otherwise we can just let B — A. We construct a {k, 0(l/A:))-magical graph 
G = {X,Y:E) with \X\ = n and \Y\ = life in 0{n) time by Lemma [2.41 with the additional property that 
each vertex in X is of degree two. We use G in the algorithm in Figure [O] to obtain an m x life matrix 
B over F. Since each vertex of X is of degree two, each entry of A is related to two entries in B. We can 
represent B by listing the value and position of its nonzero entries without handling duplicate positions, 
i.e. each nonzero entry in A introduces exactly two entries in B. Therefore, \B\ = 2\A\ and B can be 
constructed in 0(|yl|) field operations. The probability that min{rank(A), /c} = min{rank(i?), fc} is at least 
l-0(l/fc)-0(fc/|F|) by LemmaEH D 

The first part of Theorem 11.11 follows . 

Theorem 2.6. Suppose an m x n matrix A over a field F is given with m < n. There is an algorithm 
to compute min{rank(A), fc} for a given k < m in 0{\A\ + min{fc", fc|A|}) field operations with success 
probability at least 1 — 0(1/71^/"^). Furthermore, there is an algorithm to compute r — rank(v4) in 0{\A\ logr + 
min{r", r|A|}) field operations with success probability 1 — 0{l/n^i^). Each field operation can be done in 
0(logn + log |F|) steps. 

Proof. We can assume that |F| = U,{n'^) by Lemma [^?T] We also assume that k > n^/'^] otherwise if fc < n^^^ 
we just reset k to be n^/^ . We apply Theorem l2.5l to compress the mxn matrix A into an m x 0{k) matrix B. 
Then min{rank(B), fc} == min{rank(A), fc} with probability at least 1 - 0(l/fc) - 0{k/\F\) = 1 - Oil/n^/^) 
since n^^^ < k < n and |-F| — fl{n'^). And B can be constructed in 0(|A|) field operations and \B\ = 0{\A\). 
We then apply Theorem [23] again on B^ to compress the m x 0(fc) matrix B into an 0(fc) x 0(k) matrix C. 
Then min{rank(C), fc} = min{rank(i3), fc} with probability at least 1 — 0{l/n^^^) and C can be constructed 
in 0(|>1|) field operations with \C\ = 0{\A\). Now we can compute rank(C) in 0{k'^) field operations by using 
fast matrix multiplication [B]. Alternatively, we can compute rank(C) in 0(fc|C|) = 0(fc|A|) field operations 
using the black box approach in [37l[20]. Thus min{rank(v4), fc} can be computed in 0{|A| +min{fc'^, fc|^|}) 
field operations with success probability 1 — 0(1/^^/"^), since k^ < n < \A\ for fc < n^^^. To compute 
rank(A), we can simply apply the above algorithm with fc — rt^/"^, 2?i^/'^,4ri^/'^, . . . , 2'°^" n^/^ until the 
algorithm returns an answer smaller than fc or A is of full rank. Let r = rank(j4) . The failure probability 
is bounded by 0{l/n^^^) since sum of 1/fc is less than 2/n^^^. The number of field operations needed is 
0(|yl|logr + min{r",r|j4|}) since sum of fc" is 0{r'^) and sum of k\A\ is 0(r|A|). If the field size is |F|, 
then each field operation can be done in 0(log|F|) steps using fast arithmetic algorithms by Lemma I^TT] 
Since we assume \F\ = ^{n'^), each field operation can be done in 0(logmax{|i^|, n'^}) = 0(log \F\ + log7i) 
steps. n 

We can improve Theorem [2]6] slightly to reduce the time complexity to 0(min{| A| log r, nm}+inm{r'^,r\A\}) 
field operations. This is done by computing the compressed matrices aggregately and we omit the details 
here (see Section |A] for such a statement using superconcentrators). 

Finally, we find a set of min{rank(^), fc} linearly independent columns of A by applying the compression 
algorithm iteratively to reduce the number of columns of A progressively. The bounded degree condition of 
magical graphs is important in the following algorithm. 

Theorem 2.7. Suppose an m x n matrix A over a field F is given. There is an algorithm to find a set of 
min{rank(A), fc} linearly independent columns of A for a given k in 0{{\A\ + fc'^) logn) field operations with 
success probability at least 1 — 0{{\ogn)/n^'^), while each field operation can be done in 0(logn + log |_F|) 
steps. 



Proof. The idea is to compress the matrix A to a smaller matrix B, and then we can focus on columns in A 
that corresponds to a set of linearly independent columns in B. We will show that this reduce the number 
of columns of yl by a constant factor, so that we can repeat this procedure to reduce the number of columns 
of A to a small number. 



We assume k > n^^^ as in Theorem l2.6l Let c = 11. If to > ck, we first apply the algorithm in Theorem 
to A^ to compress A into a ck x n matrix A' in 0(|A|) field operations. We claim that if a set of columns 
is linearly independent in A then it is linearly independent in A' with high probability. Let 5 be a set 
of linearly independent columns of A. By Theorem 12.51 we have rank(AJ^^, g) = rank(ylr„ji 5) — \S\ with 

probability at least 1 — 0(l/n^/'^), and thus S* is a set of linearly independent columns in Al . Henceforth we 
use the smaller matrix A! to find the linearly independent columns of A. 

We then use the algorithm in Theorem 12.51 to compress A into a ck x ck matrix B in 0(|yl'|) — 0{\A\) 
field operations, while min{rank(A), fc} — min{rank(_B), fc} with probability at least 1 — 0{l/n^^^). Since 
B is a ck X ck matrix, we can directly find a set S of min{rank(i?), /c} linearly independent columns in 
B in 0{k'^) field operations using fast matrix multiplication [6]. Let G = {X,Y;E) be the magical graph 
used in the compression algorithm with \X\ — n and |y| = ck. Let T be the set of columns in A that 
correspond to the neighbors of the vertices corresponding to S in G. By the bounded degree condition 
of G, each vertex corresponding to a column in S is of degree at most 2|X|/|F| = 2n/{ck) and hence 
\T\ < 2n\S\/{ck) < 2n/c < n/5. Observe that the ck x \T\ submatrix A'j^, j, of A' is of rank at least 
min{rank(A), fc}, since the column space of S* in i? is spanned by the column space of A'^, rp. Thus we 
have reduced the original problem to finding a set of min{rank(y4.'^, t)i ^} linearly independent columns in 
a ck X {n/5) matrix A'j^, j,. We can repeat the above algorithm until the number of columns is reduced to 
0{k). Since each time we can reduce the number of columns by a constant factor, we need to repeat the 
algorithm at most O(logn) times. So the whole algorithm can be done in at most 0((|^| + A:'^)logn) field 
operations, and the failure probability is at most 0{{\ogn)/in}^^). D 

Theorem 11.11 follows from Theorem 12.61 and Theorem 12.71 



3 Dynamic Matrix Rank Algorithm 

In this section we present a dynamic algorithm for computing matrix rank and prove Theorem 11.21 Given 
an TO, X n matrix A, we will first show that rank(^) — laakiyAV) with high probability for an n x ttt. random 
Vandermonde matrix V with one variable. Then we show that the special structure of V can be used to 
update the matrix rank of A efficiently. 

Lemma 3.1. Let m < n. Let V be a n x m random Vandermonde matrix with one variable, i.e., Vij — x'^ 
for 1 < i < ri,l < j < m. Suppose x is chosen uniformly randomly in F, then for any m x n matrix A over 
F, we have rank(yl) — Tank{AV) with probability at least 1 — 0{nrn^ /\F\). 

Proof. We will first prove the lemma when A is of full rank. Suppose A is of full rank, then there exist m 
linearly independent columns. Let B = {I C [n] \ \I\ = to, det(A[„] /) 7^ 0} be the set of subsets of indices 
whose columns are linearly independent. Then B 7^ 0. By the Cauchy-Binet formula, 

det{AV)= J2 det(A[,„],/)det(V},[„]) 

I'^[n],\I\=m 

= ^det(yl[™]j)det(V7j™]). 



Now view det(V7,[,„]) as a polynomial in x. Suppose / = {ii, 12, ■ ■ ■ im\ with ii < 12 < ■ ■ ■ < im- Let Sm be 
the set of permutations of [m] . Note that 



det(V/,[™]) = ^ sgn(7r) J| T/,,,^, 

ireS,„ k=l 

m 

= ^ sgnW n x'-'^''- 
_ V^ sgn(7r)x^"=i''''''^''' 



By the rearrangement inequality X]fe=i *fe'''fe ^ X]fe=i *fe ' ^i ^^^ ^^e equality holds only when ttj, = A; for all 
k. Therefore, 

m 

deg(det(V7,[„,]))=^Zfc./c. (3.1) 

fe=i 

Clearly deg(det(Ay)) < max/gg deg(det(V/, [„])). We are going to show that the equality actually holds, by 
arguing that max/ggdeg(det(V/ [„])) is attained by only one /. Suppose not, let J ^ iiT be two sets in B 
satisfying deg(det(Vj,[,„])) = deg(det(W_[„])) = max/ee deg(det(V/_[,„])). Let j = min{i | i e {J-K)\J(K- 
J)}, and without loss of generality assume j G J. It is well know that the sets in B are the bases of a (linear) 
matroid .38. . Therefore, by the base exchange property of a matroid ([38^, Theorem 39.6), there exists some 
k & K such that (J — {j})U{fc} e B. By the choice of j, we have j < k, and thus deg(det(V(j_{j})u{fc},[m])) > 
deg(det(Vj[m])) by p.ip . contradicting the maximality of J. In particular, since B ^ %^ this implies that 
deg(det(Ay)) > and thus is a non-zero polynomial. And deg(det(V/jm])) = X^fcLi 'ik ■ k < nm? for any /. 
Therefore det(AV^) is a non-zero polynomial with total degree at most nm? . By the Schwartz-Zippel lemma, 
by substituting x with a random element in F, we have det(Al/^) 7^ and thus rank(AT^) ~ rank(A) with 
probability at least 1 — 0{nm? /\F\). 

In general let rank(yl) — k and assume without loss of generality that the first k rows of A are linearly 
independent. Clearly, rank(AV^) < rank(yl) as the column space of AV is spanned by the column space of A. 
We prove that rank(AF) > rank(A) with high probability. Let A' be the kxn submatrix oi A consisting of the 
first k rows of A, and V be the nxk submatrix of V consisting of the first k columns of V . Then by the above 
argument we have that det(A'V"') 7^ with probability at least 1 — 0{nm^ /\F\). Observe that A'V' is equal 
to the k X k submatrix (AT^)[;j] j^,] of AV . Therefore, we have rank(AV^) > rank((Ay)[s.] [j.]) ~ k — rank(A) 
with probability at least 1 — 0{ninn? /\F\). D 

The matrix AV can be computed efficiently using fast arithmetic algorithms: The multiplication of one row 
of A with V is equivalent to the evaluation of a polynomial over ra points (a;, a;^, . . . , x™) and this can be 
implemented efficiently using the following result. 

Theorem 3.2 (^ |20| Corollary 10.8). There exist an algorithm, that evaluates a degree n polynomial f £ F[x] 
at m points in F, and it takes ©(nlognloglognlogrn) field operations. 



Therefore, the matrix AV can be computed in 0(nm log n log log n log rn) field operations. By Lemma |3. 11 
to guarantee a high success probability, it is enough to work on a field with 0(n'*) elements, so that each 
field operation can be done in O(logn) steps [201. This gives an alternative method to compute rank(A) of 
anrnxn matrix in 0(nmlogm(logn)^ ~\-m^ logn) steps, which is slower than the algorithm in Theorem ll.il 
but has similar running time as previous algorithms. 

The following is an outline of the dynamic algorithm for computing matrix rank. Given an m x n matrix 
A with m < n, we generate a random n x m Vandermonde matrix V and we know by Lemma 13.11 that 
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rank(A) = rank(Ay) with high probabihty. We reduce AV to the rank normal form by elementary row and 
column operations, and maintain the decomposition that XAVY = D where X and Y are m x m invertible 
matrices and D = (^q?,)- Then rank(^) = r with high probability. We briefly describe how to maintain 
the rank under different operations. If we do a rank one update on A (i.e. A ^ A + uv^ where u and 
V are column vectors), then it corresponds to a rank one update on D and we can bring it back to the 
rank normal form efficiently. If we add a column to or delete a column from A, then we can add a row 
to or delete a row from V so that rank(Ay) is still equal to rank(j4) with high probability, because of the 
structure of Vandermonde matrices. If we add a row to or delete a row from A^ then we can do some rank one 
updates to maintain the structure of V and rank(^y) = rank(^) with high probability. The most interesting 
case is when rn < n is changed to ?n > n or vice versa. In this case we can change the decomposition of 
D = XAVY to D = {XV-^)VA{VY) and set the new X to be XV'^ and the new Y to be VY, and this 
can be implemented efficiently by fast Fourier transform and fast inverse Fourier transform. 

Lemma 3.3. Given an m x n matrix A over a field F , there is a data structure that maintains rank(^) 
supporting the following operations. 

1. The data structure can be initialized in 0(?nn log to log n log log(TO+ri) + (min{r7T,, n\)^) field operations. 

2. rank(74) can he updated in 0{mn) field operations if a rank one update is performed on A. 

3. rank(74) can he updated in 0(TO,n(logmin{?7i, ri})^) field operations if a row or a column is added to or 
deleted from A. 

The data structure requires space to store 0{mn) elements in F. The prohahility of failure in any operation 
is at most 0(71712^ /\F\), where h is the maximum n throughout the updates. 

Proof The data structure stores six matrices X € F™^", A € F™x"^i/ g pnxm^^ ^ prnxm^y £ 
pmxm^j-) g pmxm^ j^^^ j^ ^ j^y jf j^j < „ and B = VA H m > n. In the following we assume that 
m < n, when to, > n all the procedures are done in a symmetric manner. We maintain D = XBY with the 
following properties. 

1. X, y are invertible. 

2. 1^ is a Vandermonde matrix, i.e., Vij — gj for some gi £ F. 

3. _D is a matrix in the form D ~ ( {j' n) where /^ is an r x r identity matrix. 

Initialization: We choose g uniformly randomly in F and set Vij — g^^ . We can reduce B = AV into 
the rank normal form in 0{m'^) field operations, and thus obtain X and Y such that X and Y are both 
invertible and XBY = (^p"^ q) where r is the rank of B (see e.g. Proposition 16.13 of [7 ). This completes 
the initialization. Note that computing the z-th row of B is equivalent to doing a multipoint evaluation of 
a degree n polynomial with coefficients defined by the i-th row of A on the points g, g^, . . . , g™. Thus each 
row can be computed in O(nlognloglognlogm) field operations by Theorem 13. 2[ and the total cost for 
computing B is 0(n7TT.logTO,logT7-loglogri) field operations. 

Rank one update: Let A' = A + u'v'^ where u' e F'"^i and v' e i^"^i. Then XA'VY = D + 
{Xu'){v''^VY) = D + uv^ where u = Xu' and v'^ = v''^VY, and u and v can be computed in O(m^) 
and 0{nm) field operations respectively. And B' = A'V = AV + u'v'^^V can be updated in 0{nm) field 
operations. Suppose u^ ^ where Ui is the i-th entry of u. Let Ei = I — [ci — {l/ui)u)ef where e^ is 
the i-th standard unit vector. Then Ei is invertible and EiXA'VY is a sum of a diagonal matrix, at most 
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one nonzero row and at most one nonzero columrl]]. Hence we can use 0{m) elementary row and column 
operations to transform EiXA'VY into a matrix with each row and column having at most one non-zero 
entry, where the only nonzero entry is one. This matrix can be further transformed to the rank normal 
form D' = ( n' n ) by using two permutation matrices to permute the rows and columns, where r' is the 
rank of XA'VY. Let E2 be the composition of elementary row operations done, E3 be the composition of 
elementary column operations done. Pi be the permutation of rows and P2 is the permutation of columns. 
Then PiE2EiXA'VYE3P2 = D' . Note that X' = P1E2E1X and Y' = YE3P2 can be computed in 0{m'^) 
field operations. This is because Ei and E2 are compositions of 0{m) elementary operations and each 
elementary operation acting on an m x to matrix can be done in 0{m) field operations. Also permutations 
of rows and columns can be done in 0{m^) field operations. Now we have X'A'VY' = D', where A' is 
updated in 0{nm) field operations, and X' , Y' and D' are updated in 0{m^) field operations. 

Adding a column or adding a row: To add a column or a row, we can first add a zero column or a zero 
row and then do a rank one update. Since we know how to do rank one updates, we restrict our attention 
to adding a zero column and adding a zero row. Suppose we add a zero column in the end. Then we set 
A' = {A,0) and V^_^_i ■ — g^^ where c is the smallest index such that g^ 7^ Vi^i for 1 < i < n. Adding a 
zero column in the i-th column of A is done similarly by adding a new row in the i-th row of V . Then we 
maintain that B = A'V' and D = XBY . Suppose we add a zero row in the end. Then we set A' = ( 0^ ) i and 
set F/^+i = (V"/i)'"+^ for ah i. Then we update B' = A'V = ( ^ '^^^j'+i ) in 0{nm) field operations where 
V^^i is the (to + l)-th column of V . Note that ( -^ ? ) ( ) ( 1 ) = ( ) and the difference between B' 
and ( (^ ) is a single column, which is a rank one matrix. By the same argument used in rank one update, 
we can update X , Y and D in 0{m?) field operations accordingly. If the zero row is not added at the end, 
we can first permute the rows so that the added row is at the end, by updating X with XP where P is the 
corresponding permutation matrix, and then do the above procedure. Then we maintain that B' = A'V' 
and D' = X'B'Y'. 

Deleting a column or deleting a row: To delete a column or a row, we can do a rank one update to 
set the column or row to zero, and then delete a zero column or a zero row. So we restrict our attention 
to deleting a zero column or a zero row. Deleting a zero column is done by deleting the corresponding row 
in V. There is no change to X, B,Y,D, and we maintain B = A'V' and D = XBY. Suppose we delete a 
zero row at the end of A to obtain A'. Then we delete the last column of V to obtain V'. Let B' = A'V'. 
Note that B" = ( ^' q ) ^^'^ ^ ^ ( ^ ^0" ) '^here Vm is the m-th column of V, and so the difference is 
a rank one update. So we can find X", Y" , and D" such that X"B"Y" = D" where D" = ({j °). Now 
let X', Y', and D' be obtained by deleting the last row and column of X", Y" and D" respectively. Then 
X'B'Y' =. D', because D'^^ = Y.Z1 YZi KkK^C, = ELY YZ\' KkK^U = ^1. fori < * < m - 1 
and 1 < j' < TO,— 1 as Bk^i — ii k — m or I = m. Clearly the updates can be done in 0{nm) field operations. 
If the zero row deleted is not at the end, we can first permute the rows so that the deleted row is at the 
end, by updating X with XP where P is the corresponding permutation matrix, and then do the above 
procedure. 

Changing representation: Note that in the above operations we assume m < n. Some operations require 
0(771^) field operations and thus if m, > n this is greater than 0{mn). Instead we will maintain B — V A 
and D ~ XBY when m > n. To change the representation, when m = n, we rewrite D = XAVY = 
{X{V')-^)V'A{VY), and set X' = X{V')-^ and Y' = VY and V^^ = g'^ for all l<ij <n. Note that VY 
can be computed in 0{n^ log^ n log log n) field operations by Theorem 13. 2i since it is equivalent to n points 
evaluation of n degree n polynomials. Moreover, after reseting V^' = g"^^ , using Theorem l3.4l we can compute 



^ The details are as follows: Eiuv = uv + {et — {l/ui)u)uiv — meiv is a matrix with only one row is 
non-zero. If i > r, then EiXA'VY ^ Ei{D + uv"^) = EiD + u^etv'^ = D + ute^v"^ since ej D = 0. If i < r, then 
EiXA'VY = EiD + UiCiv"^ = D - (e, - {l/ui)u)ej + UiCiV^ if i < r since eJ D = ef . In either case EiXA'VY is a 
sum of a diagonal matrix, at most one nonzero row and at most one nonzero column. 
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X{V')'^^ in 0(n^ log^ n log log n) field operations, as it is equivalent to doing n points interpolation n times 
for each pairs of rows of X and X{V')~^. Also B' — V'A can be computed in 0(n^ log nloglogn) field 
operations by the multipoint evaluation algorithm in Theorem 13.21 Therefore, we maintain B' = V'A and 
X' B'Y' ~ D, and this can be used to support the above operations in a symmetric manner. 

Theorem 3.4 ( [20] Corollary 10.13). There is an algorithm that takes n points {xi^y-i) e F'^ as input and 
returns a polynomial f € F[x] with degree less than n which satisfies f{xi) — yi for each i. The algorithm 
takes 0(nlog nloglogn) field operations. 

Error probability: The rank query will only fail when rank(A) ^ rank(AV^) at some point, which happens 
with probability at most 0{nm'^/\F\) by Lemma [3. II This completes the proof. D 

Let Q be an upper bound on the number of updates to the matrix. Then n < n + Q. By setting \F\ = 
0((n + Q)^m^), then the probability that the algorithm does not make any error in the whole execution 
is at least 1 — 0(l/((n + Q)m)), while each field operation requires 0(log((n + Q)m)) steps. This proves 
Theorem O 

4 Applications 

In this section we will show some applications of Theorem 11.11 and Theorem 11.21 to problems in numerical 
linear algebra, combinatorial optimization, and dynamic data structures. In each subsection we will state 
the problems, describe the previous work, and present the improvements. 

4.1 Numerical Linear Algebra 

Let A be an TO X n matrix over a field F. Let r = rank(A). The rank-one decomposition of A is to write A 
as the sum of r rank one matrices. The null space of A is the subspace of vectors for which Ax = 0, and the 
problem is to find a basis of the null space of A. The matrix multiplication problem is to compute AB for two 
nxn matrices A and B. We will show that these problems can be solved faster when r is small. The previous 
best known algorithms require Q{nmr'^~^) field operations, where the bottleneck of these algorithms is in 
finding a set of r linearly independent columns. Note that previous randomized algorithms for computing r 
cannot be used to solve these problems, as they do not find a set of r linearly independent columns. In the 
following we assume that \F\ — J7(m + n) and |yl| = il{m + n). 

Rank-one decomposition: Without loss of generality we assume m < n; otherwise we consider A'^ instead 
of A. By Theorem 1 1.1[ we can find a set of r independent columns of A in 0((| A| +r'^) log n) field operations, 
with success probability at least 1 — 0(logn/n^/'^). Let T C [n] be a set of r independent columns, and 

5 C [to] with 15*1 = r be the set of rows such that As.t is of full rank. Again by Theorem ll.il we can find 
S in 0(1 A| + r'^) field operations with success probability at least 1 ~ 0(logn/n^/'^). Now set B = A[m],T 
and C = A^'^s,T x As,[n]- Then C[r],T = Ir and thus {BC)[m],T = A[jn],T- Similarly {BC)s,[n] = ^S,[n], 
and thus the entries of BC and A match in the rows of S and also the columns of T. Note that both 
BC and A are of rank r, and both BCs,t and As,t are of full rank. So for any i ^ S and j ^ T, 
det{Asu{i},Tu{j}) = and thus Aij is uniquely determined by other entries of A. The same applies to 
BC and thus A — BC. Clearly C can be computed in 0(r'^(i'i>i°g>-")) field operations. Thus the overall 
complexity is 0{{\A\ + r'^) logn + ^."(lajog^n)-) g^jj operations. 

Null space: By the above algorithm for rank-one decomposition, we can find S C [m], T C [n], B € pmxr^ 
and C e F^^^ such that A = BC, \S\ = \T\ — r and Cs,t — Ir, with required probability and time 
complexity. Note that Ax — ■^=> BCx = ■^=^ Cx ~ since the columns in B are linearly 
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independent. Since C[r].T = Ir, we have Cx = <=> xt — —C[n]^TX[n]-T- Thus the entries of a;[„]-T can 
be arbitrarily assigned and then the entries of xt is uniquely determined. Assume without loss of generality 
that T — {1,. . . ,r}. Then a basis {bi} for i £ [n] — T would be bi{k) = —Ck,i for 1 < fc < r, and then set 
bi{i) = 1 and set bi{j) = otherwise. 

Matrix multiplication: Applying the rank-one decomposition algorithm to A to find A = A1A2 for 
some Ai e F"^'', A2 € F'^''" in 0(|A| + r"(i'i^i°g-")) field operations. Now A2B can be computed in 
0(n"('°s„ r.1,1)^ ggy operations, and so do Ai{A2B) since a;(l, c, 1) = a;(c, 1, 1) [25]. So the overall complexity 
is 0(7i"('°S"'^'i'i) f^gy operations. 

4.2 Graph Matching 

Given an undirected graph G = (V, E), the maximum matching problem is to find a set of maximum number 
of vertex disjoint edges in G. The time complexity of the fastest combinatorial algorithms for this problem 
is 0{y/opt ■ \E\) [32j 42] [21], where opt denotes the size of a maximum matching. 

There is an algebraic formulation for the maximum matching problem proposed by Tutte [41) . Let V — 
{1, . . . , n} and Xe be a variable for each edge e. Let A be an n x n matrix where a^- = Xe and Uji = —Xe 
a e — ij € E and Uij = aji = otherwise. Tutte [41) proved that G has a perfect matching if and only if 
A is non-singular, and Lovasz [3D] generalized it to show that rank(A) — 2opt. Using the Schwartz-Zippel 
lemma, Lovasz [30) also proved that rank(A) is preserved with high probability, if we substitute non-zero 
values for the variables Xe from a sufficiently large field, say of size G(n^). This implies that the size of a 
maximum matching can be computed in 0(n") field operations, where each field operation can be performed 
in O(logn) steps. With additional non-trivial ideas, Mucha and Sankowski [33] and Harvey [H] showed how 
to also find a maximum matching in 0{n'^) field operations. This is faster than the combinatorial algorithms 
when the graph is dense and the opt is large, for example when \E\ = 8(n^) and opt = n the combinatorial 
algorithms require Q{n^-^) steps. 

We prove the statement about graph matching in Theorem 11.41 Suppose k is given and the task is to find a 
matching of size min{A;,opt}. Let k' = 2min{fc,opt}. We can first use the algorithm in Theorem 1 1.1 1 to find 
a set S of k' linearly independent columns in A in 0(|A| -I- (fc')'^) — 0{\E\ + (fc')'^) field operations, where 
\E\ is the number of edges in G. Let Ay^s be the n x k' submatrix formed by these independent columns. 
We can apply the algorithm in Theorem 1 1 . 1 1 again on Ay^s to find a set R of k' linearly independent rows in 
Av,s in 0(|Ay,s| -I- (fc')") == 0{\E\ + (fc')") field operations. Let Ar^s be the k' x k' submatrix formed by 
these rows and columns. Consider Ards^rus which is a matrix with size at most 2k' x 2k' and rank at least 
k' . Note that it is the algebraic formulation for the maximum matching problem in G[RUS], where G[RUS] 
denotes the induced subgraph on the vertices corresponding to RU S. And so there is a matching of size 
fc'/2 = min{fc, opt} in G[R U S]. We can use the algorithm of Mucha and Sankowski [33 or Harvey [H] to 
find a matching of size min{fc,opt} in 0{k'^) field operations. Thus the overall complexity is 0(|F| -I- fc") 
and this proves the statement about graph matching in Theorem ll.4l To find a matching of size opt, we can 
first use a linear time 2-approximation greedy algorithm to find a matching M of size at least opt/2, and 
then set k = 2\M\ and run the above algorithm. 

We mention two problems where this matching result can be applied. One is the maximum subset matching 
problem considered by Alon and Yuster |3] , which asks what is the maximum number of vertices in S* C V^ 
that can be matched in a matching of G. They proved that this maximum number is equal to rank(A5_y) 
where Asy is the submatrix of the Tutte matrix formed by the rows of S. Thus we can use Theorem 1 1.1 1 to 
obtain an 0(|(5(S')| -f IS*]") algorithm where \5{S)\ counts the number of edges with one endpoint in S and 
another endpoint hiV—S. This improves upon their result which takes 0(|5(S')| • |5'|''^~^"^) steps when 
|^('S')| > |5'|''^+^''/^. Another is the maximum matching problem in a lopsided bipartite graph G — (X, Y] E) 
where one side is much larger than the other side [5], that is |X| <C |F|. In this case opt < |X| and our 
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algorithm can find a maximum matching in Od-El + |-^|") steps. 

4.3 Linear Matroid Intersection and Linear Matroid Parity 

In the hnear matroid intersection problem, we are given two r x n matrices M and A^ where the columns in 
Af and N are indexed by {1, . . . , n}, and the task is to find a set S C {1, . . . , n} of maximum size so that 
the columns in S are linearly independent in both Af and N. In the linear matroid parity problem, we are 
given an r X 2n matrix where the columns are partitioned into n pairs, and the task is to find a maximum 
cardinality collection of pairs so that the union of the columns of these pairs are linearly independent. 

For the linear matroid intersection problem, Gabow and Xu |19j gave a combinatorial algorithm (using fast 
matrix multiplication) with time complexity 0(nr(opt)^/(^~'^)) = 0{nr (opt)'^'^^) when w w 2.38. Harvey [22] 
gave an algebraic algorithm with time complexity 0(nr"^^), which is faster for any opt > r'^-^'^ when 
Lu « 2.38. For the linear matroid parity problem, Gabow and Stallmann [18 gave a combinatorial algorithm 
(using fast matrix multiplication) with time complexity 0(nr"^^(opt)), and Cheung, Lau and Leung |11] 
gave an algebraic algorithm with time complexity 0(nr"^^) by extending Harvey's algorithm. 



We prove the statement about linear matroid intersection and linear matroid parity in Theorem 11.41 The 
linear matroid parity problem is a generalization of the linear matroid intersection problem, and any algo- 
rithm for the linear matroid parity problem implies an algorithm for the linear matroid intersection with the 
same time complexity, and so we only consider the linear matroid parity problem in the following. Let A be 
an r X 2n matrix where the columns are {ci, C2, . . . , C2n-i,C2n} and (c2i-i, C2i) is a column pair for 1 < i < n. 
Suppose k is given and the task is to find min{fc, opt} pairs of columns so that the union of the columns of 
these pairs are linearly independent. We use the algorithm in Theorem 12.51 to compress the matrix A into 
a 0{k) X 2n matrix A' in 0(|A|) field operations, and let the columns of A' be {c[,c'2, ■ ■ ■ , C2„_i, C2„}. Let 
k' = min{fc, opt}. Assume without loss of generality that the columns in 5 = {ci, C2, . . . , C2fc'-i, C2fc'} are lin- 
early independent. We claim that the columns in S' — {c'j^, c'2 . . . , c'2f.,_i, Cj^./} are linearly independent with 
high probability. Consider the submatrix Aij 5 of A where R is the set of all r rows of A. By Theorem [23] we 
have that rank(Afl^s) = rank(^'^, g,) with high probability where R' is the set of all 0{k) rows of A'. Since 
Tank{Afj^s) = 2fc', it implies that rank(A'^, g,) = 2k' and thus the columns in S' are linearly independent 
with high probability, proving the claim. Therefore we can apply the algorithm in [11] to solve the matroid 
parity problem on A', and this can be done in 0(nA:"~^) field operations since A' is a 0{k) x 2n matrix. 
This proves the statement about linear matroid intersection and linear matroid parity in Theorem ll.4l 

To find a solution of size opt, we can set fc = 2, 4, 8, . . . , 2'°^^ »" and apply the above algorithm until there is 
no solution of size k or there is a solution of size r. A direct implementation of this idea gives an algorithm 
to find an optimal solution in 0{\A\ log opt -|- n(opt)'^~^) field operations. We can slightly improve this to 
0(min{|A| log opt, nr} -l-n(opt)"^^) field operations by computing the compressed matrices aggregately, but 
the details are omitted here. Since opt < r, our algorithm is faster than the algorithms in [T9[|22| . 

4.4 Linear Matroid Union 

In the linear matroid union problem, we are given an r x n matrix A with r < n, and the task is to 
find a set of maximum number of disjoint bases, where a basis is a set of maximum number of linearly 
independent columns, and two bases are disjoint if they do not share any column. For example, the problem 
of finding a set of maximum number of edge disjoint spanning trees in an undirected graph is a special case 
of the linear matroid union problem. Let opt be the maximum number of disjoint bases in A, and b be 
the number of columns in a basis. Cunningham [15| gave a combinatorial algorithm with time complexity 
0{nrb{opt) + nb'^ (opt)'^) . 
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There is a well known reduction from the linear matroid union problem to the linear matroid intersection 
problem [38]. Suppose k is given and the task is to find k disjoint bases of A or determine that none exist. 
Let M be the kr x kn matrix 
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where denotes the r x n all zero matrix. Let iV be an n x A;n matrix {1,1, ... , I) where / is the n x n 
identity matrix. Then it can be checked that A has k disjoint bases if and only if the linear matroid 
intersection problem for M and N has a solution of size kb. A direct application of Harvey's algorithm [21] 
for linear matroid intersection gives an algorithm with time complexity 0{{kn) ■ (kr)^~^ + {kn) ■ n^~'^) = 
0{nr"-^k'^ +n'^k). 

To do better, we can first reduce the matrix A into a matrix with kb columns before running a linear 
matroid intersection algorithm, as follows. We can use a compact algebraic formulation for linear matroid 
intersection [22 where B = X]i=i ^i ■ rrii ■ nf where Xi is a random element from a sufficiently large field 
(say of size 0(n^)) and nii and rii are the i-th column of M and N respectively. For this particular M and 
N , we have 
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where each column of Ai is equal to the corresponding column of A multiplied by an independent random 
field element. Using the result for linear matroid intersection, it can be shown that if A has k disjoint bases, 
then B has rank kh with high probability. Furthermore, if we find kb linearly independent columns in B, 
then the corresponding columns in A can be partitioned into k disjoint bases. So, one can first find a set of 
kb linearly independent columns in B in 0{n(kr)'^~^) field operations by Gaussian elimination (or conclude 
that there are no k disjoint bases if none exist), and then delete the other columns and consider the linear 
matroid union problem for the r x kb submatrix of A. Then we can run the linear matroid intersection 
algorithm ^ to find the k disjoint bases in 0{{kb)r'^^'^k'^ + {kbyk) = 0{r'^~^bk'^+^) field operations by 
using n = kb and b < r. This results in an 0{nr'^~^k'^^^ + r'^^^bk'^^^) algebraic algorithm for the linear 
matroid union problem using existing techniques, although it was not explicitly stated in the literature. 



We are now ready to prove the statement about linear matroid union in Theorem 11.41 First we use the 
compression algorithm in Theorem l2.5l to reduce A to a 0{b) x n matrix A' with \A'\ = 0{\A\) in 0(|A|) field 
operations. By the same argument used in linear matroid parity, we can show that if A has k disjoint bases, 
then A' has the same k disjoint bases with high probability. We construct the 0{kb) x n matrix B' as in the 
previous paragraph in 0(fc|A'|) — 0{k\A\) field operations since \A'\ — 0{\A\). Then we use the algorithm in 
Theorem 1 1.11 to find kb linearly independent columns in B' in 0(fc|yl| + (kb)'^) field operations since kb < n 
(or conclude that there are no k disjoint bases if none exist). As stated in the previous paragraph, the 
corresponding kb columns in A' can be partitioned into k disjoint bases with high probability. So we delete 
other columns and only consider the 0{b) x kb submatrix A" of A' . Now we have reduced the linear matroid 
parity problem for an r x n matrix A to the linear matroid union problem for a 0{b) x kb matrix A" . We 
can run Harvey's linear matroid intersection algorithm using the above reduction to find the k disjoint bases 
in 0{{kb)b'^-'^k'^ + {kb^k) = 0{b'^k'^+'^) field operations by putting n = kb and r = 0{b). Alternatively, 
we can use Cunningham's algorithm to find the k disjoint bases in 0{{kb)b'^k + {kb)b'^k^) = 0{b^k^) field 
operations by putting n = kb and r = 0{b). Therefore the total complexity is 0(fc|A| + min{6"fc'^+^, b^k'^}) 
field operations where \A\ < nr, proving the statement about linear matroid union in Theorem ll.4l To find 
the maximum number of disjoint bases, we can use doubling (fc = 2, 4, 8, . . . , 2^°^°^') and then binary search. 
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and apply the above algorithm as in linear matroid parity, and obtain an algorithm with time complexity 
0(logopt(nr(opt) + min{&"(opt)'^"'""'^, 6^(opt)^)) field operations. Ignoring polylog factors, this is faster than 
the previous algorithms for any values of r, b, opt, n. 

4.5 Dynamic Edge Connectivities 

In this section we show that the dynamic matrix rank algorithm in Theorem II .21 can be applied to obtain an 
efficient dynamic algorithm for computing all pairs edge connectivities in a simple directed graph G = {V, E), 
supporting the operations of adding and deleting edges. The s-t edge connectivity is defined as the size of a 
minimum s-t cut, or equivalently the number of edge disjoint directed paths from s to t. 

We will use a recent algebraic formulation that relates edge connectivities to matrix ranks [T^. Construct 
an \E\ X \E\ matrix M as follows: 

Cij if the head of Ci is equal to the tail of Cj 

otherwise 
The matrix has the following properties: 

Theorem 4.1 f |12|). The s-t edge connectivity is equal to the rank of the submatrix M~^ gout (^g\ gin u\, where 
(5*"(w) and (5°"*(w) are the set of incoming and outgoing edges of v respectively. In addition, if we substitute 
random values to xtj from a field F, the claim still holds with probability at least 1 — 0(|-Ep/|i^|). 

This formulation implies an 0(|£'|") time algorithm for computing all pairs edge connectivities for simple 
directed graphs. We are going to show that using the dynamic matrix rank algorithm, we can support 
each adding and deleting edge operation in 0(|i?p) time, by maintaining the ranks of all the submatrices 

(M-i)5o>.t(^)_5i„(t) dynamically. 

First we consider the case of adding an edge. Let G be the original graph, and G be the graph with an edge 
added to G. Let M and M be the edge connectivity matrix formulation for G and G respectively. Observe 
that M is obtained from M by adding one extra row and one extra column at the end. We will maintain 
M~^ and the ranks of its submatrices, by first adding a trivial row and column to M, and then fill in the 
required entries. Let 

Since we only have to modify the last row and last column of M' to get M, we can write M = M' + UV^ 
for two {\E\ + 1) X 2 matrices U = (e, c) and V — (r, e), where c is the new column with \E\ + 1 entries, r^ 
is the new row with \E\ + 1 entries, and e be the column vector with the first |_E| entries to be zero and the 
last entry to be one. The following result shows that such a low rank update can be computed efficiently. 

Theorem 4.2 (Sherman-Morrison- Woodbury [44]). Suppose that matrices M and A/ + UV"^ are both non- 
singular, then {M + [/V^^)-i = M-i - M-^U{I + V^ M-^U)-^V'^ Ivr^ . 

By the theorem M~^ is also a rank-2 update to (Af)^^, and M~^ can be obtained from Af~^ in 0(|i!^p) 
time since U and V are (\E\ -I- 1) x 2 matrices. Similarly, any submatrix {M~^) go^t ii^\gin (^i\ can be obtained 
by a rank-2 update to {{M')~^)gouti^g-^gini^i^, and ((Af' )~-'^)i-out(s) .jin^^) can be obtained by adding at most 
one row and one column to {M^^)goutu\ginu\. Since both operations are supported by our dynamic matrix 
rank algorithm in 0(|(5°"*(s)||5™(t)|) field operations, we can maintain Tank{{M~^) go^t f^g-^gin ^^•^) and thus the 
s-t edge connectivity between any pair of vertices s and t in 0(|^°"*(s)||5*"(i)|) field operations after an 
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edge is added. Thus we can maintain all pairs edge connectivities in 0(^^ ^gy |(5°"*(s)||(5*"(i)|) = OdE'p) 
field operations. Let Q be an upper bound on the number of edge updates throughout the whole algorithm. 
For one pair, by the result in Section [31 the probability that the algorithm makes some mistake during the 
whole algorithm is at most 0{l/{Q\E\^^)), if the field size is Q{\E\^Q^^). Therefore, the probability that the 
algorithm makes a mistake for some pair during the whole algorithm is at most 0{\/{Q\E\)). Therefore, 
each field operation can be done in 0(log(|-E|Q)) steps. 

The case of deleting an edge is almost the same. Assume we are deleting the edge that correspond to the 
last row and column of M . We first write zero to all the entries of that row and column except keeping 
^'^\E\,\E\ — — Ij and then we delete the last row and column. These two steps correspond to a rank-2 update 
followed by a row and column deletion on M~^ using the dynamic matrix rank algorithm. This is just the 
reverse process for adding an edge. By the same argument as above, the new inverse and the ranks of all 
the required submatrices can be updated in 0{\E\^) field operations, where each field operation can be done 
in 0{\og{\E\Q)) steps. Note that we just require 0{\E\^ \og{\E\Q)) space to store the inverse of M. This 
proves Theorem 1 1.5 1 
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A Matrix Rank Algorithm by Superconcentrators 

In this section we present an algorithm to compute the rank of an m x n matrix with m < n in 0{mn + m") 
field operations using a superconcentrator. 

Definition A.l (Superconcentrator). A superconcentrator is a directed graph G = {V^E) with two given 
sets I QV and O C y with \I\ = \0\ = n, such that for any subsets S* C / and T Q O with \S\ =^ \T\ = k, 
there are \S\ = \T\ vertex disjoint paths from S to T . 

There exist superconcentrators with the following properties: (1) there are 0{n) vertices and 0{n) edges, 
(2) the indegrees and the outdegrees are bounded by a constant, and (3) the graph is acyclic. Moreover, the 
construction of such a superconcentrator can be done in linear time [5S] . A superconcentrator can be used 
to obtain an efficient compression algorithm. 

Lemma A. 2. Given an m x n matrix A over a field F and an integer k < min{n, m}, there is an algorithm 
to construct an m x k matrix B over F in Oinm) field operations, such that rank(i3) = min{rank(A), fc} 
with probability at least 1 — nm/\F\. 

Proof. To construct the matrix B, we first construct a superconcentrator G = {V,E) with |/| = \0\ = n in 
linear time. Add a source vertex s and add edges from s to each node in / in G. We call these edges input 
edges. Add a sink vertex t and edges from each node in O to t in G. We call these edges output edges. 
Now we associate each edge e € E with an m-dimensional vector Vg in F. The n vectors ilsu corresponding 
to the input edges are set to be the column vectors of A. Next, for each node u G V — {s,t}, for each 
incoming edge xu and each outgoing edge uy, associate the pair of edges {xu, uy) with a random coefficient 
C{xu,uy) G F. Now we process the nodes of G in a topological order. For each node u € V — {s,i}, set 
each vector associated with the outgoing edge Vuy to be J2xu£E '^{xu,uy)Vxu- Finally we choose the vectors 
associated with the first k output edges to be the column vectors of B and output the matrix B. 

Since the indegrees and the outdegrees are bounded by a constant, the number of field operations required 
to process one node in G is 0{m). Therefore the algorithm takes 0{nm) field operations. 

We analyze the probability that rank(i?) = min{rank(A), fc}. Let k' = min{rank(yl), fc}. Clearly rank(i?) < 
k' since the column space of B is spanned by the column space of A and B has only k columns. So we only 
need to show the other direction. Assume without loss of generality that A[fe/] [^.z] is of full rank. By the 
property of the superconcentrator G, there exists k' vertex disjoint paths from the first k' input nodes to the 
first k' output nodes. Set c/xu,uy) = 1 if the edges xu and uy belongs to one of the paths, and ci^u.^y\ — 
otherwise. Then all edges in the path containing the j-th input node is associated with the j'-th column 
vector of A. Thus the k' x k' submatrix i?[fc'i j/t'i of the output matrix B is the same as A, up to permutation 
of columns, and thus it is of rank k' . Therefore we can conclude that with non-zero probability the above 
algorithm outputs a matrix B with rank(i?) > rank(_B[fc/] [/;/]) = k' . Finally, note that for a fixed input A, 
each entry in the output matrix B is a multivariate polynomial with total degree 0{n) (which is the length 
of a longest path in G) with variables Ci^ei,e2)- Therefore the determinant of the first k' columns of _B is a 
multivariate polynomial of total degree 0{nk') — 0{nm). By the Schwartz-Zippel lemma, if we substitute 
the variables with random elements in F, the probability that the determinant of i?[fc'],[A;'] is nonzero and 
thus rank(i?) > k' is at least 1 — 0{nm/\F\). D 

Compared with the algorithm in Theorem 1 1 . II using magical graphs, this algorithm has the advantage the 
compressed matrix is of size k x k rather than of size 0{k) x 0[k). 

Also we can obtain an algorithm to compute r — rank(^) in 0{mn + r'^) field operations as follows. First we 
apply Lemma lA. 21 on A and get an rax n output matrix B, and then apply Lemma lA. 21 on B"^ and an get 
an n X TO output matrix G. By Lemma IA.2| the resulting matrix G has the property that the rank of any 

21 



kx k submatrix is equal to min{rank(A), k} with probability at least 1 — 0{nm/\F\). Therefore, to compute 
rank(A), one can set fc = 2, 4, 8, . . . and compute the rank of any kx k matrix of C until the returned rank is 
less than k. The total complexity of this algorithm is only 0{mn + r") field operations where r — ia.nk{A), 
which is slightly faster than the 0(|A| logr + r") algorithm stated in Theorem [ 
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